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1. Introduction 

The Fourier coefficients of modular forms encode very interesting arithmetic data. For example, divisor 
sums, partition numbers, trace of Frobenius of the reduction modulo primes of an elliptic curve over Q, and 
more generally, trace of Frobenius of many Galois representations of dimension 2 over finite fields (this being 
a conjecture of Serre) are all known to be, or conjectured to be, Fourier coefficients of modular forms. A 
particularly important family of modular forms are the so-called Hecke eigenforms. These are modular forms 
that are also simultaneous eigenforms for an algebra of operators called the Hecke operators that operate 
on the spaces of modular forms. The Fourier coefficients of Hecke eigenforms are particularly important 
arithmetically. Indeed, many of the examples given above arise as Fourier coefficients of Hecke eigenforms. 

In this article we are concerned with the computational complexity of computing the Fourier coefficients 
of these Hecke eigenforms. Currently, there are three approaches to computing the Fourier coefficients of 
modular forms: a method based on computing theta series of lattices Piz80 ; the method of modular symbols 
|Mer94[ [SteOO ; and one based on the Selberg Trace formula (see |Cha06j and Chapter 5 of [Cha05] ) . All of 
these methods result in algorithms with exponential running time to compute the Fourier coefficients. The 
Fourier coefficients of Hecke eigenforms are multiplicative and satisfy recurrences for prime powers. Since 
there are subexponential time algorithms for factoring integers, the interesting problem is to compute the 
p-th Fourier coefficient, for prime p, efficiently. However, this problem is still open in general. For any fixed 
eigenform of weight 2 one can use Schoof's algorithm ( Sch85j) for counting points on elliptic curves over 
finite fields to compute the p-th Fourier coefficient efficiently. Recent work of Edixhoven et al. suggests that 
this approach generalizes to compute eigenforms of weig ht k > 2 ( [ECd.T+06] ). 



There have been no hardness results known for computing the Fourier coefficients of eigenforms (except 
for those of Eisenstein series where the hardness results follow from [BMS86J). In this article, we give 
evidence that computing Fourier coefficients of the Hecke eigenforms for composite indices is no easier than 
factoring integers. More precisely, we show that the existence of a polynomial time algorithm that, given 
n, computes the n-th Fourier coefficient of a (fixed) Hecke eigenform implies that we can factor most RSA 
moduli (numbers that are products of two distinct primes) in polynomial time. In particular, our result 
implies a hardness result for computing the Ramanujan tau function. 

1.1. Preliminaries and notation. Since there are a number of excellent references for modular forms we 
refrain from reproducing the definitions here. Instead, we refer the reader to any of the books |Ser70|, IShi71, 
Lan76, Kob93l [Qno04[ [DS05] for the definition and background on modular forms. The Fourier expansions 
of modular forms that we refer to are the Fourier expansions at the cusp oo. In the Fourier expansions q 
stands for e 27Tlz . The letters p and q will be used for (rational) prime numbers, the latter will be used when 
no confusion can arise with the g-expansions. 

2. The Reduction 

Let Sk{To(N), x) be the space of cusp forms of ewer0 integer weight k (> 2), level N, and character \ mod N. 
In what follows, fix f(z) = Xa<n a ( n )<?" e Sk(To(N),x) to be a normalized (a(l) = 1) Hecke eigenform. 
We will also assume that / is not of CM type in the sense of Ribet |Rib77j . This means that there does 
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^The condition of even weight is needed for a technical reason in the proof. 



not exist an imaginary quadratic field, L, such that a(p) = for all primes p that are inert in L. Under 
these assumptions, a beautiful theorem of Serre ( Ser81] Corollary 2 to Theorem f 5) gives us bounds on the 
number of primes p for which a(p) = 0. 

Theorem 2.1. Let f{z) = X)i<n a ( ri )9™ ^ Sk(Fo(N), x) > 1) be a normalized Heche eigenform that is 
not of CM type. Define Pf(x) — ${p < x : p a prime such that a(p) — 0}. Then 



P f^ = ° a ^5 f ° r aU 6 > °- 

V (logo;) 2 V 

Moreover, if one assumes the Generalized Riemann Hypothesis, we have 

P f (x) = O (xi) . 
The assumption that / not be of CM type is necessary, see Remark 



The Fourier coefficients of a normalized Hecke eigenform need not be integers, but they are at least algebraic 
integers (see |Ono04j §2.4 & §2.5; the result also follows from |Shi71j Theorem 3.52). Furthermore, we know 
that each eigenvalue lies in a number field of degree at most dim Sk(Fo(N), x)f(N) since the characteristic 
polynomials of the Hecke operators have degree dim Sk(To(N), \) over the field Q(x)- In fact, the field 
Kf = Q(a(2), a(3), • • • , a(n), • • • ) is a number field and so a finite degree extension of Q. Since / is fixed 
we can assume that we can do computations in this field efficiently. We assume that the supposed algo- 
rithm that computes the Fourier coefficients takes as input an integer n and gives us the (monic) minimal 
polynomial of the n-th Fourier coefficient a{n). We also assume that the algorithm provides a complex 
approximation to a(n) that distinguishes a{n) from its conjugates. In other words, we not only have the 
sub-field Q(a(n)) C Kf, but we also have an embedding of Q(a(n)) in C. Since the space Sk{To(N), x) is 
fixed, and x is a Dirichlet character mod N , we can also compute x( n ) f° r any integer n. 

Next, we describe how we can factor RSA moduli if we can compute the Fourier coefficients a(n). We are 
given a positive integer n = pq, where p, q are distinct odd primes. We can also assume (without loss of 
generality) that gcd(A r , n) = 1. Let x = a{p)/x{p)p^ r ~ and y — a{q) j 'x(q)q~^~ ■ Note that we can also 
assume that x( n ) ^ for otherwise gcd(n, N) ^ 1. We will make the assumption that x ^ and y ^ in 
the following analysis. 

Using the algorithm to compute the Fourier coefficients of / we can compute 

A = dc f n 2 X y 
a(n) 
X(n) 

and 

B = dcf a(n 2 ). 

Now by multiplicativity and the recurrences for prime powers that a(n) satisfy we have ( Kob93] III. §5) 

(1) B = a(n 2 ) = a(p 2 )a(q 2 ) 

(2) = (a(p) 2 p^xip^aiq) 2 - q k - X X {q)) 

(3) =n k - 1 x(n)(x 2 -l)(y 2 -l). 

Thus we have a pair of simultaneous equations for x and y which we can solve. Setting a = A/n^~ and 
(3 = B /n k ~ 1 x{n), one obtains 



2 a 2 - 13 + 1 ± J (a 2 - + l) 2 - 4a 2 
x = 



and 



Substituting the definitions of a and (3 and clearing denominators we get 
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(A 2 X {n) - B + n fe - 1 x(n)) ± \J {A 2 X (n) - B + n k - 1 x(n)) 2 - 4A 2 n k - 1 X (n) 2 



2x(n)n k 



k-l 



We note that the radicand is the square of an algebraic integer (see below) and hence the square root can be 
computed exactly. This can be computed efficiently by computing numerical approximations to the square 
roots of all the conjugates of radicand. By the definition of x we have that 



2 a (p) 

x - 



x(p) 2 p k 1- 

Note that this quantity is not zero under our assumption that x ^ 0. We claim that x 2 cannot be an 
algebraic integer if p is large enough. For otherwise, since fc — 1 is odd, this would make ypp an element 
°f Q(Xj a (2), a(3), • • • , a(n), ■ ■ ■ ), but the latter is a finite extension and thus if p is large enough it cannot 
contain y/p. Thus we can recover p from the above expression by taking the gcd of the denominator of 
the above expression with n. Since the quantity is an algebraic number the (reduced) denominator in the 
expression is the leading coefficient of the minimal polynomial over Z. 

Suppose x = but y ^ (i.e. a(p) — but a(q) ^ 0), we can still proceed as follows. By equation ([3]) we 
find that B = n fe_1 x(fi)(l — y 2 ). Thus we can still get y 2 and by the above argument find q. 

Thus our reduction will succeed in factoring the integer n, unless both a(p) and a(q) are zero. Since the set 
of such primes is density (by Theorem 1 2.1 |) , we get the following theorem: 

Theorem 2.2. Let f{z) = Xa<n a ( n )9™ ^ S2k(^o{N), x) be a normalized Hecke eigenform that is not 
of CM-type. Suppose there is a polynomial time algorithm that computes a(n) given n. Then there is a 
polynomial time algorithm that factors a density 1 subset of the RSA moduli. 

In the case that / £ Sk(To(N),x) an d k is odd the entire reduction works as long as p~^~ does not divide 
a(p) for one of the primes dividing n. The failure of the reduction occurs very rarely. Indeed, if k > 3 and 
k is odd then this implies that a(p) = mod p which means that p is a, so called, non- ordinary prime. A 
heuristic argument given in |Gou97| shows that the number of non-ordinary primes below x is 0(logloga;). 
Thus, it is likely that the result of Thcorcm l2.2l remains true even for odd weight cuspidal eigenforms. 



Example 2.3. We illustrate the reduction in the case of the Ramanujan Tau function r(n) that gives the 
Fourier coefficients of A, a weight 12 eigenform of level 1 and trivial character (see 



Let n = 15, from the tables in [Leh43j one sees that r(15) = 1217160 and r(15 2 ) = 2897808426675. In the 
notation of the proof of the theorem we have 



81288256 

a 



2 - and 



474609375' 
1431016507 

~ 4271484375' 

From this one finds that x 2 = and gcd(1953125, 15) = 5. 

Example 2.4. The space S , 4(ro(29)) with trivial character has a newform, / (say), whose expansion begins 
q + iq 2 + (— 37 — 8)<7 3 + (— 27— 7)q A + (47 — l)q 5 + • • • , where 7 is a root of x 2 + 2x~ 1. A short computation in 
MAGMA ( |BC03j ) tells us that the 15th Fourier coefficient is -57-4 and that the 225th Fourier coefficient 
is -268O7 - 6168. MAGMA computes that either 

x 2 = ^(30 7 + 73) or x 2 = ^(-^ + 17) 

corresponding to the two square roots of (a 2 — j3 + l) 2 — 4a (again we have preserved the notation used in 
the proof). In any case, the denominators in these expressions yield a proper factor of 15. 
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Remark 2.5. The hardest cases of factoring the RSA moduli are believed to be those of the form pq where 
the primes p and q are both approximately the same size. One might wonder if the set of RSA moduli on 
which our reduction works includes such numbers also. This is indeed true. The number of RSA moduli 
below a bound x that have both the factors being approximately the same size can be estimated as follows. 
Let c be a constant with < c < 1. Using the prime number theorem the number of RSA moduli pq for 
which Cy/x < p,q < y/x is 

( - k( c Vx)\ _ n f x \ 

{ 2 ) H Ug 2 J' 

where 7r is the prime counting function. Meanwhile, the number of RSA moduli with primes of the same 
size for which our reduction fails is bounded above by (using Theorem 12.11) 

and if we assume the GRH this upper bound can be strengthened to 0(x 3 / 4 ). Thus, our reduction does 
indeed work on a density 1 subset of the "interesting" RSA moduli. 

Remark 2.6. For CM-forms, the prime indexed Fourier coefficients vanish for, roughly, half the primes. And 
our reduction will fail if both the prime factors of n are divisible by such primes. 

2.1. Computing a basis of cusp forms. Theorem 12.21 has the following consequence for the problem of 
computing any basis of cusp forms (with algebraic Fourier coefficients) for Sk(To(N), x). 

Corollary 2.7. Fix N, a positive integer, k > 2, an even integer, and \ a Dirichlet character modulo N . 
Assume that Sk(To(N),x) contains a Hecke eigenform of non-CM type. Fix also a basis given by the Fourier 
expansion 

fi = ^2 a l (m)q m for 1 < i < d, 

l<m 

such that the a.j (m) 's are algebraic. Suppose there is a polynomial time algorithm that, given n, computes the 
list of Fourier coefficients, <2j(m), for 1 < i < d, then there is a polynomial time algorithm that can factor a 
density 1 subset of the RSA moduli. 

Proof : By our assumption there is a Hecke eigenform not of CM-type in 5fc(Fo(iV), x). This form can be 
normalized by taking a scalar multiple, call this normalized eigenform g. Now, since g belongs to Sk(To(N), \) 
and the /, span the space we must have that g = X)i<i<d a «/»i where 014 are algebraic numbers. The n-th 
Fourier coefficient of g is Yli<i<d Ct i a i( m )i an d so this can be computed (in polynomial time) using the 
supposed algorithm for computing the <Zj(m)'s. The result now follows from Theorem 12.21 □ 

We now investigate the conditions under which the assumption made in Corollary 12.71 (that Sk (To (N) , x) 
contains an eigenform of non-CM type) holds. A construction due to Hecke |Hec37) (also described by 
Shimura) shows how one can obtain essentially all the eigenforms of CM-type (see |Rib77j §3). This con- 
struction together with dimension formulas for Sk(To(N), x) can be used to show the existence of eigenforms 
of non-CM type. The results of Theorem 3.5 and Corollary 3.5 of |Rib77j summarize the construction of CM 
forms by Hecke. Essentially, these results imply that one gets CM-forms corresponding to quadratic imagi- 
nary fields of discriminant D dividing the level N, and each ideal class character of the orders of discriminant 
N in these fields. From this observation and bounds on class numbers of imaginary quadratic fields, we find 
that the number of eigenforms of CM-type in Sk(To(N), x) is bounded above by JV5 +£ for every e > 0. The 
dimension of Sk(To(N), x) (see |CQ77j ) on the other hand is Q(kN). Furthermore, the space Sk(To(N), x) 
has a basis of eigenforms; thus, if N and k are large enough there will always be eigenforms in Sk(To(N),x) 
that are not of CM-type. In other words, for large enough k and TV, the assumption made in Corollary 12.71 
holds. Consequently, computing a basis for such spaces is at least as hard as factoring RSA moduli. 
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2.2. The Ramanujan Tau function. The Ramanujan Tau function r(n) is denned to be the n-th Fourier 
coefficient of the Discriminant function A(z) : 

A(z) = q Y[(l-q n r, 

l<n 

= 1 - 24q 2 + 252q 3 - 1472g 4 + 4830<7 5 - 6048g 6 - 16744g 7 + • ■ ■ 

=def J2 T ^ qU - 
l<n 

It is a fact that A is a Hecke eigenform of weight 12 and level 1. There are no CM forms of level 1 (since 
the discriminant of the underlying CM field must divide the level), so A is not a CM form. Moreover, a 
conjecture of Lehmer states that r(n) is never zero. If we assume Lehmer's conjecture then the proof of our 
result now yields a slightly stronger conclusion: 

Corollary 2.8. Assuming Lehmer's conjecture, computing the Ramanujan tau function is at least as hard 
as factoring RSA moduli. 

Acknowledgement: The authors would like to thank Tonghai Yang for pointing them to Hecke's con- 
struction of CM forms. 
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